Kernel Regression Trees

This paper presents a novel method for learning in domains with continuous target variables. The method integrates regression trees with kernel regression models. The integration is done by adding kernel regressors at the tree leaves producing what we call kernel regression trees. The approach is motivated by the goal of trying to take advantage of the different biases of the two regression methodologies. The presented method is implemented. Kernel regression trees are comprehensible and accurate regression models of the data. Experimental comparisons on both artificial and real world domains revealed the superiority of kernel regression trees when compared to the two individual approaches. The use of kernel regression at the trees leaves gives a significant performance gain. Moreover, a good performance level is achieved with much smaller trees than if kernel regression was not used. Compared to kernel regression our method improves its comprehensibility and execution time.